Integrated Sequence Tagging for Medieval Latin Using Deep Representation Learning

In this paper we consider two sequence tagging tasks for medieval Latin: part-of-speech tagging and lemmatization. These are both basic, yet foundational preprocessing steps in applications such as text re-use detection. Nevertheless, they are generally complicated by the considerable orthographic variation which is typical of medieval Latin. In Digital Classics, these tasks are traditionally solved in a (i) cascaded and (ii) lexicon-dependent fashion. For example, a lexicon is used to generate all the potential lemma-tag pairs for a token, and next, a context-aware PoS-tagger is used to select the most appropriate tag-lemma pair. Apart from the problems with out-of-lexicon items, error percolation is a major downside of such approaches. In this paper we explore the possibility to elegantly solve these tasks using a single, integrated approach. For this, we make use of a layered neural network architecture from the field of deep representation learning. Introduction and challenges The Latin language, and its historic variants in particular, have long been a topic of major interest in Natural Language Processing [Piotrowksi 2012]. Especially in the community of Digital Humanities, the automated processing of Latin texts has always been a popular research topic. In a variety of computational applications, such as text re-use detection [Franzini et al, 2015], it is desirable to annotate and augment Latin texts with useful morpho-syntactical or lexical information, such as lemma’s. In this paper, we will focus on two sequence tagging tasks for medieval Latin: part-ofspeech tagging and lemmatization. Given a piece of Latin text, the task of lemmatization involves assigning each word to a single dictionary headword or ‘lemma’: a baseform label (preferably in a normalized orthography) grouping all word tokens which only differ in spelling and/or inflection [Knowles et al, 2004]. The task of lemmatization is closely related to that of part-of-speech (PoS) tagging [Jurafsky et al, 2000], in which each word in a running text should be assigned a tag indicating its part of speech or word class (e.g. noun, verb, ...). The difficulty of PoS-tagging strongly depends of course on the complexity and granularity of the tagset chosen. Lemmatization and PoS-tagging are classic forms of sequence labeling, in which tags are assigned to words, both on the basis of their individual appearance, as well as the other words which surround them. While both lemmatization and PoS-tagging are rather basic preprocessing steps, they are generally complicated by a number of interesting challenges which the Latin language poses. First of all, while plain stemming might take us a long way [Schinke et al, 1996], many Latin suffixes cannot be automatically linked to an unambiguous morphological category. Words ending in –ter, for example, correspond to no less than six different parts of speech: nouns (fra-ter), adjectives (dex-ter), pronouns (al-ter), adverbs (gravi-ter), numeral adverbs (qua-ter) and prepositions (in-ter) [Manuel de lemmatisation, LASLA, 2013]. Additionally, like many other languages, Latin is teeming with homographs which require context to be disambiguated. A token such as legi can both be lemmatized under the verb lego as under the noun lex. Similarly ambiguous tokens include common forms such quae, satis or venis. For lemmatization specifically, another problem are verb forms which show no resemblance to their lemma at all. The fact that tuli is an ‘active 1st person singular perfect’ of fero is not at all obvious, and the same goes for fero’s perfect participle latus, which could in its own turn be confused with the homonymous common noun latus (“side”). A tagger has to learn the morphological connection between tuli, latus and fero by moving beyond outward appearances (prefixes, word stems or suffixes), and by properly modelling the immediate context surrounding these words.